Skip to content

OTA-1585: Add conformance test suites to the CVO tests extension #1238

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Oct 24, 2025
Merged

OTA-1585: Add conformance test suites to the CVO tests extension #1238

merged 2 commits into from
Oct 24, 2025

Conversation

@DavidHurta
Copy link
Contributor

@DavidHurta DavidHurta commented Sep 30, 2025 •
edited
Loading

Add two popular conformance test suites to the CVO tests extension.

Imported tests based on the qualifiers will be selected by the respective test suite and will be invoked on the execution of the suite's parent suite after the integration is completed.

Follow up PRs will:

Running the openshift/cluster-version-operator/conformance/parallel suite does execute the sole test as expected:

$ ./_output/linux/amd64/cluster-version-operator-tests run-suite openshift/cluster-version-operator/conformance/parallel
  Running Suite:  - /home/dhurta/GolandProjects/cluster-version-operator
  ======================================================================
  Random Seed: 1761320947 - will randomize all specs

  Will run 1 of 1 specs
  ------------------------------
  [Jira:Cluster Version Operator] cluster-version-operator-tests should support passing tests
  /home/dhurta/GolandProjects/cluster-version-operator/test/cvo/cvo.go:9
  • [0.000 seconds]
  ------------------------------

  Ran 1 of 1 Specs in 0.000 seconds
  SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 0 Skipped
[
  {
    "name": "[Jira:Cluster Version Operator] cluster-version-operator-tests should support passing tests",
    "lifecycle": "blocking",
    "duration": 0,
    "startTime": "2025-10-24 15:49:07.914082 UTC",
    "endTime": "2025-10-24 15:49:07.914615 UTC",
    "result": "passed",
    "output": ""
  }
]

Running the openshift/cluster-version-operator/conformance/serial suite does not execute anything as expected (there are no such tests at the moment):

$ ./_output/linux/amd64/cluster-version-operator-tests run-suite openshift/cluster-version-operator/conformance/serial
null

@DavidHurta DavidHurta mentioned this pull request Sep 30, 2025
Draft
@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Sep 30, 2025
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Sep 30, 2025

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 30, 2025
@DavidHurta
test: Create conformance parallel test suite
8e312d4 
The sole existing test will be included in the test suite.

For more information regarding test labels, see [1, 2]

[1]: https://github.com/openshift/origin/tree/main/test/extended#test-labels
[2]: Integration Guide for OpenShift Tests Extension
@DavidHurta
test: Create conformance serial test suite
3736449 
For more information regarding test labels, see [1, 2]

[1]: https://github.com/openshift/origin/tree/main/test/extended#test-labels
[2]: Integration Guide for OpenShift Tests Extension
@DavidHurta DavidHurta force-pushed the add-cvo-tests-to-conformance-suite branch from 76a1ed9 to 3736449 Compare October 24, 2025 15:40
@DavidHurta DavidHurta changed the title test: Add tests to conformance parallel suite OTA-1585: Add conformance test suites to the CVO tests extension Oct 24, 2025
@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Oct 24, 2025
@openshift-ci-robot
Copy link
Contributor

openshift-ci-robot commented Oct 24, 2025 •
edited by openshift-ci bot
Loading

@DavidHurta: This pull request references OTA-1585 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.21.0" version, but no target version was set.

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot
Copy link
Contributor

openshift-ci-robot commented Oct 24, 2025 •
edited by openshift-ci bot
Loading

@DavidHurta: This pull request references OTA-1585 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.21.0" version, but no target version was set.

In response to this:

Add two popular conformance test suites to the CVO tests extension.

Imported tests based on the qualifiers will be selected by the respective test suite and will be invoked on the execution of the suite's parent suite after the integration is completed.

Follow up PRs will:

  • Include the compressed test binary in the CVO's container image
  • Register the CVO tests extension in the origin repository
  • Ideally provide a rich documentation how to select the test suites, structure, etc. in the repository,

Running the openshift/cluster-version-operator/conformance/parallel suite does execute the sole test as expected:

$ ./_output/linux/amd64/cluster-version-operator-tests run-suite openshift/cluster-version-operator/conformance/parallel
 Running Suite:  - /home/dhurta/GolandProjects/cluster-version-operator
 ======================================================================
 Random Seed: 1761320947 - will randomize all specs

 Will run 1 of 1 specs
 ------------------------------
 [Jira:Cluster Version Operator] cluster-version-operator-tests should support passing tests
 /home/dhurta/GolandProjects/cluster-version-operator/test/cvo/cvo.go:9
 • [0.000 seconds]
 ------------------------------

 Ran 1 of 1 Specs in 0.000 seconds
 SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 0 Skipped
[
 {
   "name": "[Jira:Cluster Version Operator] cluster-version-operator-tests should support passing tests",
   "lifecycle": "blocking",
   "duration": 0,
   "startTime": "2025-10-24 15:49:07.914082 UTC",
   "endTime": "2025-10-24 15:49:07.914615 UTC",
   "result": "passed",
   "output": ""
 }
]

Running the openshift/cluster-version-operator/conformance/serial suite does not execute anything as expected (there are no such tests at the moment):

$ ./_output/linux/amd64/cluster-version-operator-tests run-suite openshift/cluster-version-operator/conformance/serial
null

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@DavidHurta DavidHurta marked this pull request as ready for review October 24, 2025 16:07
@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Oct 24, 2025
@openshift-ci-robot
Copy link
Contributor

openshift-ci-robot commented Oct 24, 2025 •
edited by openshift-ci bot
Loading

@DavidHurta: This pull request references OTA-1585 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.21.0" version, but no target version was set.

In response to this:

Add two popular conformance test suites to the CVO tests extension.

Imported tests based on the qualifiers will be selected by the respective test suite and will be invoked on the execution of the suite's parent suite after the integration is completed.

Follow up PRs will:

Running the openshift/cluster-version-operator/conformance/parallel suite does execute the sole test as expected:

$ ./_output/linux/amd64/cluster-version-operator-tests run-suite openshift/cluster-version-operator/conformance/parallel
 Running Suite:  - /home/dhurta/GolandProjects/cluster-version-operator
 ======================================================================
 Random Seed: 1761320947 - will randomize all specs

 Will run 1 of 1 specs
 ------------------------------
 [Jira:Cluster Version Operator] cluster-version-operator-tests should support passing tests
 /home/dhurta/GolandProjects/cluster-version-operator/test/cvo/cvo.go:9
 • [0.000 seconds]
 ------------------------------

 Ran 1 of 1 Specs in 0.000 seconds
 SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 0 Skipped
[
 {
   "name": "[Jira:Cluster Version Operator] cluster-version-operator-tests should support passing tests",
   "lifecycle": "blocking",
   "duration": 0,
   "startTime": "2025-10-24 15:49:07.914082 UTC",
   "endTime": "2025-10-24 15:49:07.914615 UTC",
   "result": "passed",
   "output": ""
 }
]

Running the openshift/cluster-version-operator/conformance/serial suite does not execute anything as expected (there are no such tests at the moment):

$ ./_output/linux/amd64/cluster-version-operator-tests run-suite openshift/cluster-version-operator/conformance/serial
null

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@DavidHurta
Copy link
Contributor Author

DavidHurta commented Oct 24, 2025 •
edited
Loading

Testing the PR

I have run the testwith command in origin to test whether the sanity test does get executed in our e2e job with all the relevant PRs:

/testwith openshift/cluster-version-operator/main/e2e-agnostic-ovn https://github.com/openshift/cluster-version-operator/pull/1236 https://github.com/openshift/cluster-version-operator/pull/1238

Not on the latest commit, but on the last commit with new changes to the actual logic.

The [Jira:Cluster Version Operator] cluster-version-operator-tests should support passing tests sanity test can be seen in the passed tests (Prow Link).

wking
wking approved these changes Oct 24, 2025
View reviewed changes
Copy link
Member

@wking wking left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Oct 24, 2025
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Oct 24, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: DavidHurta, wking

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@DavidHurta
Copy link
Contributor Author

DavidHurta commented Oct 24, 2025 •
edited
Loading

/verified-by @DavidHurta

The changes affect only tests. The changes are not visible in the CI nor in a cluster. I have tested the changes locally and in combination with other relevant PRs using testwith as mentioned in this PR. I believe this is sufficient.

@DavidHurta
Copy link
Contributor Author

/verified by @DavidHurta

See my previous comment.

@openshift-ci-robot openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label Oct 24, 2025
@openshift-ci-robot
Copy link
Contributor

@DavidHurta: This PR has been marked as verified by @DavidHurta.

In response to this:

/verified by @DavidHurta

See my previous comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@hongkailiu
Copy link
Member

/cc

@openshift-ci openshift-ci bot requested a review from hongkailiu October 24, 2025 18:28
@openshift-ci-robot
Copy link
Contributor

/retest-required

Remaining retests: 0 against base HEAD 75c15b5 and 2 for PR HEAD 3736449 in total

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Oct 24, 2025

@DavidHurta: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@openshift-merge-bot openshift-merge-bot bot merged commit 52cc813 into openshift:main Oct 24, 2025
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@wking wking wking approved these changes

@hongkailiu hongkailiu Awaiting requested review from hongkailiu

Assignees

@wking wking

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. verified Signifies that the PR passed pre-merge verification criteria

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@DavidHurta @openshift-ci-robot @hongkailiu @wking